Large-scale Dictionary Learning For Local Coordinate Coding

Dictionary learning is a method to learn dictionary items adapted to data of a given distribution. It is shown that dictionary learned from data is more suited for vision task than universal dictionaries [4]. Traditionally, Vector Quantization (VQ), or using k-means to learn data cluster centroids, is a simple and popular method in the bag-of-features framework [5]. Recently, sparse coding is used in visual dictionary learning and achieves lower reconstruction error [6]. To capture manifold geometry of the data distribution, local coordinate coding (LCC) [7] is proposed and it achieves state-of-the-arts performance on PASCAL VOC 2009 challenge. One problem with the original LCC for learning a visual dictionary is that the time complexity grows linearly with the number of samples. For large-scale datasets of millions of samples, the computational cost becomes unacceptable. In this paper, we propose an online LCC dictionary learning algorithm that only processes one or a small mini-batch of random samples at every iteration round based on [3]. This stochastic approach converges almost surely and can scales up gracefully to large-scale datasets. To learn a visual dictionary D from data samples xi, LCC optimizes the following objective function [7]